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REAL PARTY IN INTEREST 



The real party in interest in this appeal is the following party: International Business 
Machines Corporation of Armonk, New York. 
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RELATED APPEALS AND INTERFERENCES 



With respect to other appeals or interferences that will directly affect, or be directly affected 
by, or have a bearing on the Board's decision in the pending appeal, there are no such appeals or 
interferences. 
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STATUS OF CLAIMS 



A. TOTAL NUMBER OF CLAIMS IN APPLICATION 

Claims in the application are: 1-51 

B. STATUS OF ALL THE CLAIMS IN APPLICATION 

1 . Claims canceled: 9-11, 19-20, 34, 42-43 

2. Claims withdrawn from consideration but not canceled: none 

3. Claims pending: 1-8, 12-18, 21-33, 35-41 and 44-51 

4. Claims allowed: none 

5. Claims rejected: 1-8, 12-18, 21-33, 35-41 and 44-51 

6. Claims objected to: none 

C. CLAIMS ON APPEAL 

The claims on appeal are: 1-8, 12-18, 21-33, 35-41 and 44-51 



STATUS OF AMENDMENTS 

No amendments were made after the Final Office Action dated August 23, 2007. 
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SUMMARY OF CLAIMED SUBJECT MATTER 



A. CLAIM 1 - INDEPENDENT 

The subject matter of claim 1 is directed to a computer-implemented method for 
monitoring responses to test questions presented in a data processing system (100, 400, 600). 
{Specification on page 4, lines 3-26; and Figures 1, 4, and 6.) A presentation of the test questions 
is identified on the data processing system. {Specification on page 8, lines 9-19; and page 18, 
lines 16-21.) Responsive to the presentation of the test questions on the data processing system, 
test question timing data is monitored. {Specification on page 18, lines 22-32; page 22, lines 1- 
21; page 30, lines 8-10 and 16-32; and steps 1014, 1016 in Figure 10.) The test question timing 
data represents an elapsed time since an answered question from the test questions has been 
presented. (See Specification on page 34, lines 14-16.) The elapsed time is an amount of time in 
attempting to answer a test question. {Specification on page 33, lines 23-25.) An alert is 
generated after the test question timing data exceeds a threshold while continuing to present the 
test question for the test taker to answer. {Specification on page 23, lines 1-7; page 25, lines 8-14 
and 18-27; page 30, line 8, through page 31, line 14; and steps 1016, 1020, 1022, 1026 in Figure 
10.) The alert apprises a test taker that the elapsed time is excessive for the test question 
{Specification on page 25, 18-27.) The alert is generated based on an alert schedule for the test 
question. {Specification on page 29, lines 10-14; step 920 in Figure 9; and step 1022 in Figure 
10.) The alert schedule is generated for the test taker based on a customized alert profile for the 
test taker. {Specification on page 29, 8-14; and steps 918, 920 in Figure 9.) The customized alert 
profile includes previous performance information of the test taker, information to associate a 
level of difficulty of a particular test question with a capability category of the test taker to 
answer the particular test question, and alert thresholds for the test questions. {Specification on 
page 25, line 15, thorough page 27, line 8; page 28, line 9, through page 29, linel4; and Figures 
8A and 8B.) 

B. CLAIM 6 - INDEPENDENT 

The subject matter of claim 6 is directed to a computer-implemented method of 
monitoring a test question response time. {Specification on page 4, lines 3-26; and Figures 1 , 4, 
and 6.) A test is administered to a remotely located user of a client device (108, 111, 1 14 in 
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Figure 1 and 608, 612 in Figure 6) receiving test question timing data from the client device. 
(Specification on page 8, lines 9-19; and page 18, lines 16-21.) The test question timing data 
represents an elapsed time used by the remotely located user in attempting to answer a test 
question from a plurality of test questions that are to be provided to the client device during 
administration of the test. (Specification on page 25, lines 8-14 and 18-27; page 33, lines 23-25; 
page 34, lines 14-16.) The test question timing data is output to a proctor device (105-107 in 
Figure 1 and 604 in Figure 6) such that the proctor device may monitor the elapsed time in 
attempting to answer the test question for the remotely located user. (Specification on page 1 8, 
lines 22-32; page 22, lines 1-21; and 530 in Figure 5.) The remotely located user can send an 
instant message to and receive an instant message from the proctor device and the proctor device 
can send an instant message to and receive an instant message from a plurality of remotely 
located users. (Specif cation on page 18, line 32, through page 19, line 9; page 21, lines 23-31; 
page 22, lines 1-17; and 520 in Figure 5.) Instant messages are used to communicate and clarify 
test question wording details, test instructions, and the test question timing data during the test. 
(Specification on page 18, line 32, through page 19, line 9; page 21, lines 23-31; page 22, lines 1- 
17; and 520 in Figure 5.) The remotely located user is alerted when the test question timing data 
exceeds a predetermined threshold based on an alert schedule for the test question while the 
remotely located user continues to attempt to answer the test question. (Specification on page 23, 
lines 1-7; page 25, lines 8-14 and 18-27; page 30, lines 29-32; and steps 1016, 1020, 1022, 1026 
in Figure 1 0.) The alert schedule is generated for the remotely located user based on a 
customized alert profile for the remotely located user. (Specification on page 29, 8-14; steps 918, 
920 in Figure 9; and step 1022 in Figure 10.) The customized alert profile includes previous 
performance information of the remotely located user, information to associate a level of 
difficulty of a particular test question with a capability category of the remotely located user to 
answer the particular test question, and a plurality of predetermined thresholds for the plurality of 
test questions. (Specification on page 25, line 15, thorough page 27, line 8; page 28, line 9, 
through page 29, linel4; and Figures 8A and 8B.) 
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C. CLAIM 23 - INDEPENDENT 

The subject matter of claim 23 is directed to an apparatus for monitoring responses to test 
questions presented in a data processing system (100, 400, 600). {Specification on page 4, lines 
3-26; and Figures 1, 4, and 6.) The apparatus comprises identifying means (450) for identifying 
presentation of the test questions on the data processing system. {Specification on page 8, lines 9- 
19; and page 18, lines 16-21.) The apparatus comprises monitoring means (430, 500), responsive 
to the presentation of the test questions on the data processing system, for monitoring test 
question timing data in which the test question timing data represents an elapsed time since an 
answered question from the test questions has been presented. {Specification on page 18, lines 
22-32; page 22, lines 1-21; page 30, lines 8-10 and 16-32; and steps 1014, 1016 in Figure 10.) 
The elapsed time is an amount of time in attempting to answer a test question. (See Specification 
on page 34, lines 14-16.) The apparatus comprises generating means for generating an alert after 
the test question timing data exceeds a threshold while continuing to present the test question for 
the test taker to answer. {Specification on page 23, lines 1-7; page 25, lines 8-14 and 18-27; page 
30, line 8, through page 31, line 14; and steps 1016, 1020, 1022, 1026 in Figure 10.) The alert 
apprises a test taker that the elapsed time is excessive for the test question. {Specification on page 
33, lines 23-25.) The alert is generated based on an alert schedule for the test question. 
{Specification on page 29, lines 10-14; step 920 in Figure 9; and step 1022 in Figure 10.) The 
alert schedule is generated for the test taker based on a customized alert profile for the test taker. 
{Specification on page 29, 8-14; and steps 918, 920 in Figure 9.) The customized alert profile 
includes previous performance information of the test taker, information to associate a level of 
difficulty of a particular test question with a capability category of the test taker to answer the 
particular test question, and alert thresholds for the test questions. {Specification on page 25, line 
15, thorough page 27, line 8; page 28, line 9, through page 29, linel4; and Figures 8A and 8B.) 

D. CLAIM 28 - INDEPENDENT 

The subject matter of claim 28 is directed to a data processing system (100, 200, 400, 
600) comprising a bus system (206, 212, 480), a communications unit (210) connected to the bus 
system, a storage device (232, 440, 450, 470, 616, 624) connected to the bus system, and a 
processing unit (202, 204) connected to the bus system. {Specification on page 10, lines 14-26; 
page 11, lines 10-ll;page 14, lines 22-28; page 23, lines 27-3 1 ; and Figures 1,2, 4, and 6.) The 
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storage device includes a set of instructions and the processing unit executes the set of 
instructions. {Specification on page 14, linesl6-28.) The processing unit executes the set of 
instructions to identify presentation of the test questions on the data processing system. 
(Specification on page 8, lines 9-13; and page 18, lines 16-21.) The processing unit executes the 
set of instructions to monitor test question timing data in response to the presentation of the test 
questions on the data processing system in which the test question timing data represents an 
elapsed time since an answered question from the test questions has been presented. 
{Specification on page 18, lines 22-32; page 22, lines 1-21; page 30, lines 8-10 and 16-32; and 
steps 1014, 1016 in Figure 10.) The elapsed time is an amount of time in attempting to answer a 
test question. (Specification on page 33, lines 23-25.) The processing unit executes the set of 
instructions to generate an alert after the test question timing data exceeds a threshold while 
continuing to present the test question for a test taker to answer. (Specification on page 29, lines 
10-14; step 920 in Figure 9; and step 1022 in Figure 10.) The alert apprises the test taker that the 
elapsed time is excessive for the test question. (Specification on page 25, 18-27.) The alert is 
generated based on an alert schedule for the test question. (Specification on page 29, 10-14; step 
920 in Figure 9; and step 1022 in Figure 10.) The alert schedule is generated for the test taker 
based on a customized alert profile for the test taker. (Specification on page 29, 8-14; and steps 
918, 920 in Figure 9.) The customized alert profile includes previous performance information 
of the test taker, information to associate a level of difficulty of a particular test question with a 
capability category of the test taker to answer the particular test question, and alert thresholds for 
the test questions. (Specification on page 25, line 15, thorough page 27, line 8; page 28, line 9, 
through page 29, linel4; and Figures 8A and 8B.) 

E. CLAIM 29 - INDEPENDENT 

The subject matter of claim 29 is directed to an apparatus for monitoring a test question 
response time. (100, 400, 600). (Specification on page 4, lines 3-26; and Figures 1, 4, and 6.) 
The apparatus comprises a controller (410) and at least one interface (420, 430, 440, 450, 460, 
470) coupled to the controller. (Specification on page 14, line 22, through page 15, line 5; and 
Figure 4.) The controller administers a test to a remotely located user of a client device (108, 
111,114 in Figure 1 and 608, 612 in Figure 6) via the at least one interface (420, 450). 
(Specification on page 8, lines 9-19; and page 18, lines 16-21.) The controller receives test 
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question timing data from the client device via the at least one interface (420, 610, 614). 
{Specification on page 18, lines 22-24; and page 23, lines 8-13.) The test question timing data 
represents an elapsed time used by the remotely located user in attempting to answer a test 
question from a plurality of test questions that are to be provided to the client device during 
administration of the test. (Specification on page 18, lines 22-32; page 22, lines 1-21; page 30, 
lines 8-10 and 16-32; and steps 1014, 1016 in Figure 10.) The controller outputs the test 
question timing data to a proctor device (105-107 in Figure 1 and 604 in Figure 6) via the at least 
one interface (430, 500) such that the proctor device may monitor the elapsed time in attempting 
to answer the test question for the remotely located user (608, 612). (Specification on page 18, 
lines 22-32; page 22, lines 1-21; and 530 in Figure 5.) The processing unit provides instant 
messaging between the remotely located user and a proctor. (Specification on page 18, line 32, 
through page 19, line 9; page 21, lines 23-31; page 22, lines 1-17; and 520 in Figure 5.) The 
remotely located user can send an instant message to and receive an instant message from the 
proctor and the proctor can send an instant message to and receive an instant message from a 
plurality of remotely located users. (Specification on page 18, line 32, through page 19, line 9; 
page 21, lines 23-3 1 ; page 22, lines 1-17; and 520 in Figure 5.) Instant messages (520) are used 
to communicate and clarify test question wording details, test instructions, and the test question 
timing data during the test. (Specification on page 18, line 32, through page 19, line 9; page 21, 
lines 23-3 1 ; page 22, lines 1-17; and 520 in Figure 5.) The controller alerts the remotely located 
user when the test question timing data exceeds a predetermined threshold based on an alert 
schedule for the test question. (Specification on page 23, lines 1-7; page 25, lines 8-14 and 18-27; 
page 30, lines 29-32; and steps 1016, 1020, 1022, 1026 in Figure 10.) The alert schedule is 
generated for the remotely located user based on a customized alert profile for the remotely 
located user. (Specification on page 29, 8-14; steps 918, 920 in Figure 9; and step 1022 in Figure 
10.) The customized alert profile includes previous performance information of the remotely 
located user, information to associate a level of difficulty of a particular test question with a 
capability category of the remotely located user to answer the particular test question, and a 
plurality of predetermined thresholds for the plurality of test questions. (Specification on page 25, 
line 15, thorough page 27, line 8; page 28, line 9, through page 29, linel4; and Figures 8A and 
8B.) 
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F. CLAIM 46 - INDEPENDENT 

The subject matter of claim 46 is directed to a computer program product in a computer 
readable medium for monitoring a test question response time. (Specification on page 4, lines 3- 
26; and Figures 1, 4, and 6.) The computer program product provides first instructions for 
administering a test to a remotely located user of a client device (108, 111, 1 14 in Figure 1 and 
608, 612 in Figure 6). (Specification on page 8, lines 9-19; and page 18, lines 16-21.) The 
computer program product provides second instructions for receiving test question timing data 
from the client device. (Specification on page 18, lines 22-24; and page 23, lines 8-13.) The test 
question timing data represents an elapsed time used by the remotely located user in attempting 
to answer a test question from a plurality of test questions that are to be provided to the client 
device during administration of the test. (Specification on page 25, lines 8-14 and 18-27; page 33, 
lines 23-25; page 34, lines 14-16.) The computer program product provides third instructions for 
outputting the test question timing data to a proctor device such that the proctor device may 
monitor the elapsed time in attempting to answer the test question for the remotely located user. 
(Specification on page 18, lines 22-32; page 22, lines 1-21; and 530 in Figure 5.) The computer 
program product provides fourth instructions for providing instant messaging between the 
remotely located user and a proctor. (Specification on page 18, line 32, through page 19, line 9; 
page 21, lines 23-31; page 22, lines 1-17; and 520 in Figure 5.) The remotely located user can 
send an instant message to and receive an instant message from the proctor and the proctor can 
send an instant message to and receive an instant message from a plurality of remotely located 
users. (Specification on page 18, line 32, through page 19, line 9; page 21, lines 23-31; page 22, 
lines 1-17; and 520 in Figure 5.) Instant messages (520) are used to communicate and clarify 
test question wording details, test instructions, and the test question timing data during the test. 
(Specification on page 18, line 32, through page 19, line 9; page 21, lines 23-31; page 22, lines 1- 
17; and 520 in Figure 5.) The computer program product provides fifth instructions for alerting 
the remotely located user when the test question timing data exceeds a predetermined threshold 
based on an alert schedule for the test question while the remotely located user continues to 
attempt to answer the test question. (Specification on page 23, lines 1-7; page 25, lines 8-14 and 
18-27; page 30, lines 29-32; and steps 1016, 1020, 1022, 1026 in Figure 10.) The alert schedule 
is generated for the remotely located user based on a customized alert profile for the remotely 
located user. (Specification on page 29, 8-14; steps 918, 920 in Figure 9; and step 1022 in Figure 
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10.) The customized alert profile includes previous performance information of the remotely 
located user, information to associate a level of difficulty of a particular test question with a 
capability category of the remotely located user to answer the particular test question, and a 
plurality of predetermined thresholds for the plurality of test questions. {Specification on page 25, 
line 15, thorough page 27, line 8; page 28, line 9, through page 29, line 14; and Figures 8A and 
8B.) 



G. CLAIM 47 - INDEPENDENT 

The subject matter of claim 47 is directed to a computer program product in a computer 
readable medium for use in monitoring responses to test questions presented in a data processing 
system (100, 400, 600). (Specification on page 4, lines 3-26; and Figures 1, 4, and 6.) The 
computer program product provides first instructions for identifying presentation of the test 
questions on the data processing system. (Specification on page 8, lines 9-19; and page 18, lines 
1 6-2 1 .) The computer program product provides second instructions, responsive to the 
presentation of the test questions on the data processing system, for monitoring test question 
timing data in which the test question timing data represents an elapsed time since an answered 
question from the test questions has been presented. (Specification on page 18, lines 22-32; page 
22, lines 1-21; page 30, lines 8-10 and 16-32; and steps 1014, 1016 in Figure 10.) The elapsed 
time is an amount of time in attempting to answer a test question. (See Specification on page 34, 
lines 14-16.) The computer program product provides third instructions for generating an alert 
after the test question timing data exceeds a threshold while continuing to present the test 
question for the test taker to answer. (Specification on page 23, lines 1-7; page 25, lines 8-14 and 
18-27; page 30, line 8, through page 3 1 , line 14; and steps 1016, 1020, 1022, 1026 in Figure 10.) 
The alert apprises a test taker that the elapsed time is excessive for the test question. 
(Specification on page 25, 18-27.) The alert is generated based on an alert schedule for the test 
question. (Specification on page 29, lines 10-14; step 920 in Figure 9; and step 1022 in Figure 
10.) The alert schedule is generated for the test taker based on a customized alert profile for the 
test taker. (Specification on page 29, 8-14; and steps 918, 920 in Figure 9.) The customized alert 
profile includes previous performance information of the test taker, information to associate a 
level of difficulty of a particular test question with a capability category of the test taker to 
answer the particular test question, and alert thresholds for the test questions. (Specification on 
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page 25, line 15, thorough page 27, line 8; page 28, line 9, through page 29, linel4; and Figures 
8A and 8B.) 

H. CLAIM 51 - INDEPENDENT 

The subject matter of claim 51 is directed to a computer-implemented method for 
monitoring responses to test questions presented in a data processing system. (100, 400, 600). 
(Specification on page 4, lines 3-26; and Figures 1, 4, and 6.) The method comprises 
administering a plurality of tests from an examination server to a plurality of remotely located 
users on a plurality of user devices. (Specification on page 7, lines 5-16; page 8, lines 9-19; page 
20, lines 16-28; and 510 in Figure 5.) For each test of the plurality of tests that is administered, a 
session identification is established for the administration of the test to the remotely located user. 
(Specification on page 15, lines 23-31; and page 20, lines 16-28.) The session identification 
includes a user identification, a test identifier, and a proctor device identifier. (Specification on 
page 16, lines 2-14) For each test of the plurality of tests that is administered, presentation of the 
test questions is identified on a user device of the user devices. (Specification on page 8, lines 9- 
19; and page 18, lines 16-21.) Responsive to the presentation of the test questions on the user 
device, test question timing data is monitored. (Specification on page 18, lines 22-32; page 22, 
lines 1-21; page 30, lines 8-10 and 16-32; and steps 1014, 1016 in Figure 10.) The test question 
timing data represents an elapsed time since an answered question from the test questions has 
been presented. (See Specification on page 34, lines 14-16.) The elapsed time is an amount of 
time in attempting to answer a test question. (Specification on page 33, lines 23-25.) For each 
test of the plurality of tests that is administered, the test question timing data is correlated to the 
administration of the test to the remotely located user based on the session identification. 
(Specification on page 16, lines 15-24.) For each test of the plurality of tests that is 
administered, the test question timing data is output to the proctor device, based on the proctor 
device identifier, in response to determining that evidence of greater than expected response time 
to the test question is present. (Specification on page 16, lines 5-9; page 18, lines 12-27; and page 
34, lines 1-7.) For each test of the plurality of tests that is administered, generating an alert 
message after the test question timing data exceeds a threshold while continuing to present the 
test question for the remotely located user to answer. (Specification on page 23, lines 1-7; page 
25, lines 8-14 and 18-27; page 30, line 8, through page 31, line 14; and steps 1016, 1020, 1022, 
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1026 in Figure 10.) The alert message apprises the remotely located user that the elapsed time is 
excessive for the test question. {Specification on page 25, 18-27.) The alert message is generated 
based on an alert schedule for the test question. {Specification on page 29, lines 10-14; step 920 
in Figure 9; and step 1022 in Figure 10.) The alert schedule is generated for the remotely located 
user based on a customized alert profile for the remotely located user. {Specification on page 29, 
8-14; and steps 918, 920 in Figure 9.) The customized alert profile includes previous 
performance information of the remotely located user, information to associate a level of 
difficulty of a particular test question with a capability category of the remotely located user to 
answer the particular test question, and a plurality of alert thresholds for the test questions. 
{Specification on page 25, line 15, thorough page 27, line 8; page 28, line 9, through page 29, 
linel4; and Figures 8A and 8B.) For each test of the plurality of tests that is administered, the 
remotely located user can send instant messages to and receive instant messages from a proctor 
device associated with the examination server and the proctor device can send instant messages 
to and receive instant messages from the plurality of remotely located users. {Specification on 
page 18, line 32, through page 19, line 9; page 21, lines 23-31; page 22, lines 1-17; and 520 in 
Figure 5.) Instant messages (520) are used to communicate and clarify test question wording 
details, test instructions, and the test question timing data for the remotely located user during the 
test. {Specification on page 18, line 32, through page 19, line 9; page 21, lines 23-31; page 22, 
lines 1-17; and 520 in Figure 5.) 

I. CL AIM 1 7 - DEPENDENT 

The subject matter of claim 17, which depends from independent claim 6, is directed to a 
computer-implemented method further comprising monitoring the test question timing data for 
evidence of greater than expected response time to the test question, wherein outputting the test 
question timing data to a proctor device is performed by a test administration system in response 
to determining that evidence of greater than expected response time to the test question is 
present. {Specification on page 16, lines 5-9; page 18, lines 12-27; and page 34, lines 1-7.) 

J. CLAIM 21 - DEPENDENT 

The subject matter of claim 21, which depends from independent claim 6, is directed to a 
computer-implemented method further comprising storing a response to the test question from . 
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the remotely located user to update the customized alert profile for use in future tests. 
(Specification on page 24, lines 1-26.) 

K. CLAIM 22 - DEPENDENT 

The subject matter of claim 22, which depends from independent claim 6, is directed to a 
computer-implemented method further comprising storing of the timing data for the test question 
to update timing data for the remotely located user in the customized alert profile for use in 
future tests. (Specification on page 24, lines 1-26; and page 32, lines 21-22.) 

L. CLAIM 40 - DEPENDENT 

The subject matter of claim 40, which depends from independent claim 29, is directed to 
an apparatus further comprising a controller that monitors the test question timing data for 
evidence of greater than expected response time to the test question, wherein the controller 
outputs the test question timing data to a proctor device is performed by a test administration 
system in response to determining that evidence of greater than expected response time to the test 
question is present. (Specification on page 16, lines 5-9; page 18, lines 12-27; and page 34, lines 
1-7.) 

M. CLAIM 44 - DEPENDENT 

The subject matter of claim 44, which depends from independent claim 29, is directed to 
an apparatus further comprising storing a response to the test question from the remotely located 
user to update the customized alert profile for use in future tests. (Specification on page 24, lines 
1-26.) 

N. CLAIM 45 - DEPENDENT 

The subject matter of claim 45, which depends from independent claim 29, is directed to 
an apparatus further comprising storing of the timing data for the test question to update timing 
data for the remotely located user in the customized alert profile for use in future tests. 
(Specification on page 24, lines 1-26; and page 32, lines 21-22.) 
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GROUNDS OF REJECTION TO BE REVIEWED ON APPEAL 



The grounds of rejection to review on appeal are as follows: 

A. GROUND OF REJECTION 1 (Claims 1-3, 23-25, 28, and 47-48) 

The Final Office Action rejects claims 1-3, 23-25, and 47-48 under 35 U.S.C. § 103(a) as 
being unpatentable over Thomas, Method and Apparatus for Improving Performance on 
Multiple-Choice Exams . U.S. Patent No. 5,885,087 (March 23, 1999), hereinafter referred to as 
Thomas, in view of Sugimoto, Method and System for Performing Adaptive Test , U.S. Patent 
No. 6,755,661 (June 29, 2004), herein after referred to as Sugimoto. 

B. GROUND OF REJECTION 2 (Claims 4-5, 26-27, 49, and 50) 

The Final Office Action rejects claims 4-5, 26-27, 49, and 50 under 35 U.S.C. § 103(a) as 
being unpatentable over Thomas in view of Sugimoto as applied to the claims above and further 
in view of admitted prior art, hereinafter referred to as APA. 

C. GROUND OF REJECTION 3 (Claims 6, 12, 17-18, 29, 32-33, 35, 40-41, and 46) 

The Final Office Action rejects claims 6, 12, 17-18, 29, 32-33, 35, 40-41, and 46 under 
35 U.S.C. § 103(a) as being unpatentable over Thomas in view of Sugimoto and further in view 
of Hoehn-Saric et ah, System and Administration of Remotelv-Proctored, Secure Examinations 
and Methods Therefor . U.S. Patent No 5915973 (June 29, 1999), hereinafter referred to as 
Hoehn-Saric. 

D. GROUND OF REJECTION 4 (Claims 7-8, 13, 16, 30-31, 36, and 39) 

The Final Office Action rejects claims 7-8, 13, 16, 30-31, 36, and 39 under 35 U.S.C. § 
103(a) as being unpatentable over Thomas in view of Sugimoto and Hoehn-Saric as applied to 
the claims above and further in view of APA. 
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E. GROUND OF REJECTION 5 (Claims 14-15, 21-22, 37-38, and 44-45) 

The Final Office Action rejects claims 14-15, 21-22, 37-38, and 44-45 under 35 U.S.C. § 
103(a) as being unpatentable by Thomas in view of Sugimoto and Hoehn-Saric as applied to 
claim 6 above, and further in view of Kershaw et ah, System and Methods for Computer Based 
Testing , U.S. Patent No. 5,827,070 (October 27, 1998), hereinafter referred to as Kershaw. 

F. GROUND OF REJECTION 6 (Claim 51) 

The Final Office Action rejects claim 51 under 35 U.S.C. § 103(a) as being unpatentable 
by Kershaw in view of Thomas, Hoehn-Saric, and Sugimoto. 
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ARGUMENT 



A. GROUND OF REJECTION 1 (Claims 1-3, 23-25, 28, and 47-48) 

The Examiner has rejected claims 1-3, 23-25, 28, and 47-48 under 35 U.S.C. § 103(a) as 
being anticipated by Thomas, Method and Apparatus for Improving Performance on Multiple- 
Choice Exams , U.S. Patent No. 5,885,087 (March 23, 1999) in view of Sugimoto, Method and 
System for Performing Adaptive Test . U.S. Patent No. 6,755,661 (June 29, 2004). This rejection 
is respectfully traversed. 

With respect to independent claims 1, 23, 28, and 47, the Examiner states: 

4. In regards to claims 1, 3, 23, 25, 28 and 47, Thomas discloses a test timing 
system that discloses a computerized testing device that conducts testing for a user 
whereby a question is presented to the user and the time taken by the user to 
answer the question is tracked and displayed and may be compared to a 
predetermined time (25-20 and 4:45-65). Examiner contends that the constant 
display of the elapsed time constitutes an alert and that the predetermined time for 
a question used to compare to the elapsed time disclosed in the reference would 
constitute an alert threshold. Thomas further discloses that the system may be 
used to practice examination skills and improve their test taking skills (3:4-14) 
and that the system maintains player profiles in order to provide a history of the 
user's progress including performance by subject or topic (7:43-58). Thomas lacks 
explicitly disclosing that the alert schedule is based on the profile of the user's 
previous performance, the relative question difficulty, and alert thresholds and 
that presentation of test questions are based on levels of difficulty of the test 
questions and the ability of the test taker. 

5. In related prior art, Sugimoto discloses a testing system that adapts the 
timing of a test question when a user takes less than an allotted time on a question 
and provides the extra time on a later question for the user (abstract and 18:48- 
54). Sugimoto further discloses that profiles of the test taker are maintained, 
including skill level of the user (95-7 and fig 9, user ID and skill code), and the 
profile is used to determine questions presented to the test taker (917-20), a 
question database that includes information on the question difficulty to be related 
to the user's skill setting (6:5-7) and a preset time limit for each question (6:63- 
40), which examiner contends is analogous to an alert threshold, that is changed 
by the system as the user's skill is determined. One skilled in the art would 
recognize the advantages of providing more time on questions a user has trouble 
with and less time on questions the user finds easy in order to complete an exam 
in the allotted time with the most correct answers possible thus improving the test 
taker's performance and tailoring a test to a user's ability in order to help them 
improve incrementally. 

6. Therefore it would have been obvious to one skilled in the art at the time 
to have modified Thomas in view of Sugimoto to include the adaptive timing 
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system in order to further aid the test taker in completing the test in the allotted 
time while giving as much time as necessary to correctly answer questions and 
customize the tests presented to the user's ability. 

Final Office Action dated August 23, 2007, pages 2-3. 

Claim 1, which is representative of the other rejected independent claims 23, 28, and 47 

with regard to similarly recited subject matter, reads as follows: 

1 . A computer-implemented method for monitoring responses to test 
questions presented in a data processing system, the method comprising the 
computer implemented steps of: 

identifying presentation of the test questions on the data processing 

system; 

responsive to the presentation of the test questions on the data processing 
system, monitoring test question timing data in which the test question timing data 
represents an elapsed time since an answered question from the test questions has 
been presented, wherein the elapsed time is an amount of time in attempting to 
answer a test question; and 

generating an alert after the test question timing data exceeds a threshold 
while continuing to present the test question for the test taker to answer , wherein 
the alert apprises a test taker that the elapsed time is excessive for the test 
question, wherein the alert is generated based on an alert schedule for the test 
question , wherein the alert schedule is generated for the test taker based on a 
customized alert profile for the test taker , and wherein the customized alert profile 
includes previous performance information of the test taker, information to 
associate a level of difficulty of a particular test question with a capability 
category of the test taker to answer the particular test question, and alert 
thresholds for the test questions , (emphasis added) 

The Examiner bears the burden of establishing a prima facie case of obviousness based 
on the prior art when rejecting claims under 35 U.S.C. § 103. In re Fritch, 972 F.2d 1260, 23 
U.S.P.Q.2d 1780 (Fed. Cir. 1992). Thomas and Sugimoto, taken alone or in combination, do not 
teach or suggest "generating an alert after the test question timing data exceeds a threshold while 
continuing; to present the test question for the test taker to answer, wherein the alert apprises a 
test taker that the elapsed time is excessive for the test question, wherein the alert is generated 
based on an alert schedule for the test question, wherein the alert schedule is generated for the 
test taker based on a customized alert profile for the test taker, and wherein the customized alert 
profile includes previous performance information of the test taker, information to associate a 
level of difficulty of a particular test question with a capability category of the test taker to 
answer the particular test question, and alert thresholds for the test questions," as recited in 
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claims 1, 23, 28, and 47. 

Thomas is directed to a computerized learning approach that enables a user to improve 
their performance on multiple-choice exams. In general, the invention forces test-takers to 
practice their examination skills and subject matter knowledge in a systematic way. The 
systematic way forces the users to follow a proven problem-solving approach designed to 
improve their performance. Thomas ' invention provides performance feedback to users, for 
example, elapsed time per question and percentage of correctly answered questions. Figure 1 
shows user's performance information 18 in memory 10. Thomas ' invention can also pinpoint 
for the user the substantive areas of the exam, which the user is either weak or strong. Further, 
Thomas ' invention is able to assist the user in predicting his/her eventual score. Thomas 
discloses a constant display of an elapsed time. The Examiner contends that the constant display 
of the elapsed time constitutes an alert and that the predetermined time for a question used to 
compare to the elapsed time disclosed in the reference would constitute an alert threshold. 
Thomas does not mention alert schedules or alert profiles. An alert schedule indicates when 
alerts will be sent to the examinee. A constant display teaches away from this. Thomas does not 
teach or suggest "generating an alert after the test question timing data exceeds a threshold while 
continuing to present the test question for the test taker to answer, wherein the alert apprises a 
test taker that the elapsed time is excessive for the test question, wherein the alert is generated 
based on an alert schedule for the test question , wherein the alert schedule is generated for the 
test taker based on a customized alert profile for the test taker , and wherein the customized alert 
profile includes previous performance information of the test taker, information to associate a 
level of difficulty of a particular test question with a capability category of the test taker to 
answer the particular test question, and alert thresholds for the test questions ," as recited in 
claims 1, 23, 28, and 47. 

Sugimoto is directed to a method and system for performing adaptive test. A pressure 
sensitivity test is performed to examine a pressure sensitivity of a solver to time. The method 
judges, based on a result of the pressure sensitivity test, whether or not the pressure sensitivity of 
the solver Is equal to or higher than a predetermined threshold. If the pressure sensitivity is equal 
to or higher than the predetermined threshold, the solver is prompted to answer a pending 
question, acquires an input of the answer, and determines if a time period consumed by the solver 
in answering the pending question is shorter than a time limit set for answering the pending 
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question and, if shorter, gives the solver an increased time limit for answering a next question. 
Sugimoto discloses a skill-selecting program that may prompt a user for a skill field (category) 
and skill level (difficulty). Sugimoto discloses displaying a time period from the display of the 
test question, and displaying the time limit itself. A rest time period, which is calculated by 
subtracting the time elapsed from the display of the test question from the time limit for that test 
question, may also be displayed. The Examiner contends that a preset time limit for each 
question is analogous to an alert threshold, that is changed by the system as the user's skill is 
determined. Sugimoto does not mention alert schedules or alert profiles. Sugimoto does not 
teach or suggest "generating an alert after the test question timing data exceeds a threshold while 
continuing to present the test question for the test taker to answer, wherein the alert apprises a 
test taker that the elapsed time is excessive for the test question, wherein the alert is generated 
based on an alert schedule for the test question , wherein the alert schedule is generated for the 
test taker based on a customized alert profile for the test taker , and wherein the customized alert 
profile includes previous performance information of the test taker, information to associate a 
level of difficulty of a particular test question with a capability category of the test taker to 
answer the particular test question, and alert thresholds for the test questions ," as recited in 
claims 1, 23, 28, and 47. 

In claims 1, 23, 28, and 47 of the present invention, an alert is generated after the test 
question timing data exceeds a threshold that has been defined for a specific user for a specific 
question. The alert schedule is generated based on an alert profile that is customized for a 
specific user and for specific test questions. In these claims, alerts may be generated for example 
after a specific user spends 2 minutes trying to answer a specific test question, then again after 5 
minutes of trying to answer the same test question, and, in addition, an alert may be generated 
after exceeding a defined threshold equal to or greater than the time limit for the specific test 
question if each of these thresholds are defined in the alert profile for the specific user and 
specific test question. The alerts will be generated as indicated in the customized alert profile for 
the specific user and for the specific test question for that specific user. The alert notification of 
the present invention is a more intelligent and robust alert notification than merely displaying a 
constant display of elapsed time since a test question is presented. 

Thomas and Sugimoto fail to teach or suggest "generating an alert after the test question 
timing data exceeds a threshold while continuing to present the test question for the test taker to 
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answer, wherein the alert apprises a test taker that the elapsed time is excessive for the test 
question, wherein the alert is generated based on an alert schedule for the test question, wherein 
the alert schedule is generated for the test taker based on a customized alert profile for the test 
taker, and wherein the customized alert profile includes previous performance information of the 
test taker, information to associate a level of difficulty of a particular test question with a 
capability category of the test taker to answer the particular test question, and alert thresholds for 
the test questions," as recited in claims 1, 23, 28, and 47. Therefore, the alleged combination of 
Thomas and Sugimoto does not teach or suggest this feature. 

In view of the above, Appellants respectfully request withdrawal of the rejection of 
independent claims 1, 23, 28, and 47 under 35 U.S.C. § 103(a). Additionally, Thomas and 
Sugimoto, taken individually or in combination, do not teach or suggest the features of dependent 
claims 2-3, 24-25, and 48 at least by virtue of their dependency on independent claims 1, 23, and 
47, respectively. Accordingly, Appellants respectfully request withdrawal of the rejection of 
claims 1-3, 23-25, 28, and 47-48 under 35 U.S.C. § 103(a). 

B. GROUND OF REJECTION 2 (Claims 4-5, 26-27, 49, and 50) 

The Final Office Action rejects claims 4-5, 26-27, 49, and 50 under 35 U.S.C. § 103(a) as 
being unpatentable over Thomas in view of Sugimoto as applied to the claims above and further 
in view of admitted prior art, hereinafter referred to as APA. This rejection is respectfully 
traversed. 

Since claims 4, 5, 26, 27, 49 and 50 depend from independent claims 1, 23, and 47, the 
same distinctions between Thomas and Sugimoto and the invention recited in claims 1, 23, and 
47 apply to dependent claims 4, 5, 26, 27, 49 and 50. As discussed above, Thomas and 
Sugimoto, taken alone or in combination, do not teach or suggest that "generating an alert after 
the test question timing data exceeds a threshold while continuing to present the test question for 
the test taker to answer, wherein the alert apprises a test taker that the elapsed time is excessive 
for the test question, wherein the alert is generated based on an alert schedule for the test 
question, wherein the alert schedule is generated for the test taker based on a customized alert 
profile for the test taker, and wherein the customized alert profile includes previous performance 
information of the test taker, information to associate a level of difficulty of a particular test 
question with a capability category of the test taker to answer the particular test question, and 
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alert thresholds for the test questions," as recited in claims 1, 23, and 47. Accordingly, 
Appellants respectfully requests withdrawal of the rejection of claim 4, 5, 26, 27, 49 and 50 
under 35 U.S.C. § 103(a). 

C. GROUND OF REJECTION 3 (Claims 6, 12, 17-18, 29, 32-33, 35, 40-41, and 46) 

The Final Office Action rejects claims 6, 12, 17-18, 29, 32-33, 35, 40-41, and 46 under 
35 U.S.C. § 103(a) as being unpatentable over Thomas in view of Sugimoto and further in view 
of Hoehn-Saric et al, System and Administration of Remotelv-Proctored, Secure Examinations 
and Methods Therefor . U.S. Patent No 5915973 (June 29, 1999), hereinafter referred to as 
Hoehn-Saric. This rejection is respectfully traversed. 

C.l Claims 6, 12, 18, 29, 32-33, 35, 41, and 46 

With respect to independent claims 6, 29, and 46, the Examiner states: 

11. In regards to claims 6, 29, 32-33 and 46, Thomas in view of Sugimoto 
discloses the testing system described above for claim 1 where the system may 
operate over a network, which examiner is interpreting to be an interface, with a 
client (3:63-65). Thomas lacks in disclosing the use of instant messaging. 

12. In related prior art, Hoehn-Saric teaches that the administrator of a test has 
great flexibility in sending and receiving messages associated with the 
administration of a test (Col6 and 8). This flexibility may include sending and 
responding to messages with the test product users as quickly as the physical 
interconnection is capable of processing and sending them, making them "instant 
messages". One skilled in the art would recognize the advantages of providing a 
messaging system that provides rapid communication in a timed testing situation. 

13. Therefore, it would have been obvious to one of ordinary skill in the art to 
provide test examination system as disclosed by Thomas with messaging 
capability to take full advantage of the speed of the remote connection with the 
test product user to provide the ability to send and receive instant messages as 
taught by Hoehn-Saric for the purposes of distributing test evaluations to users in 
a more timely fashion in a time critical environment. 

Final Office Action dated August 23, 2007, pages 4-5. 

Claim 6, which is representative of the other rejected independent claims 29 and 46 with 

regard to similarly recited subject matter, reads as follows: 

6. A computer-implemented method of monitoring a test question response 
time, comprising the steps of: 

administering a test to a remotely located user of a client device; 
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receiving test question timing data from the client device, the test question 
timing data representing an elapsed time used by the remotely located user in 
attempting to answer a test question from a plurality of test questions that are to 
be provided to the client device during administration of the test; 

outputting the test question timing data to a proctor device such that the 
proctor device may monitor the elapsed time in attempting to answer the test 
question for the remotely located user; 

wherein said remotely located user can send an instant message to and 
receive an instant message from said proctor device and wherein said proctor 
device can send an instant message to and receive an instant message from a 
plurality of remotely located users, and wherein instant messages are used to 
communicate and clarify test question wording details, test instructions, and the 
test question timing data during the test; and 

alerting the remotely located user when the test question timing data 
exceeds a predetermined threshold based on an alert schedule for the test question 
while the remotely located user continues to attempt to answer the test question , 
wherein the alert schedule is generated for the remotely located user based on a 
customized alert profile for the remotely located user , and wherein the customized 
alert profile includes previous performance information of the remotely located 
user, information to associate a level of difficulty of a particular test question with 
a capability category of the remotely located user to answer the particular test 
question, and a plurality of predetermined thresholds for the plurality of test 
questions , (emphasis added) 

The Examiner bears the burden of establishing a prima facie case of obviousness based 
on the prior art when rejecting claims under 35 U.S.C. § 103. In re Fritch, 972 F.2d 1260, 23 
U.S.P.Q.2d 1780 (Fed. Cir. 1992). For an invention to be prima facie obvious, the prior art must 
teach or suggest all claim limitations. In re Royka, 490 F.2d 981, 180 USPQ 580 (CCPA 1974). 

Thomas, Sugimoto, and Hoehn-Saric, taken alone or in combination, do not teach or 
suggest "alerting the remotely located user when the test question timing data exceeds a 
predetermined threshold based on an alert schedule for the test question while the remotely 
located user continues to attempt to answer the test question, wherein the alert schedule is 
generated for the remotely located user based on a customized alert profile for the remotely 
located user, and wherein the customized alert profile includes previous performance information 
of the remotely located user, information to associate a level of difficulty of a particular test 
question with a capability category of the remotely located user to answer the particular test 
question, and a plurality of predetermined thresholds for the plurality of test questions," as 
recited in independent claims 6, 29, and 46. 
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As discussed above, Thomas and Sugimoto, do not teach this alerting step as recited in the 
independent claims. In addition, Hoehn-Saric does not provide for the deficiencies of Thomas 
and Sugimoto with regard to the alerting step of amended independent claims 6, 29, and 49. 
Hoehn-Saric is directed to a system for controlling the administration of remotely proctored, 
secure examinations at a remote test station, and a method for administering examinations. 
Hoehn-Saric discloses that audio/visual proctoring data can be transmitted to a central station 
and displayed so that an administrator can view the testing event. Hoehn-Saric does not mention 
alerts, alert thresholds, alert schedules, or alert profiles. 

Additionally, Thomas, Sugimoto, and Hoehn-Saric, taken alone or in combination, do not 
teach or suggest that "said remotely located user can send an instant message to and receive an 
instant message from said proctor device and wherein said proctor device can send an instant 
message to and receive an instant message from a plurality of remotely located users, and 
wherein instant messages are used to communicate and clarify test question wording details, test 
instructions, and the test question timing data during the test," as recited in amended independent 
claims 6, 29, and 46. The Office Action states that Thomas lacks in disclosing the use of instant 
messaging. Hoehn-Saric discloses that audio/visual proctoring data can be transmitted to a 
central station and displayed so that an administrator can view the testing event. Hoehn-Saric 
does not teach the instant messaging features as recited in amended claims 6, 29, and 46. For 
example, in amended claims 6, 29, and 46, the proctor device can send an instant message to a 
plurality of remotely located users to communicate and clarify test instructions. 

Thomas, Sugimoto, and Hoehn-Saric fail to teach or suggest "alerting the remotely 
located user when the test question timing data exceeds a predetermined threshold based on an 
alert schedule for the test question while the remotely located user continues to attempt to answer 
the test question, wherein the alert schedule is generated for the remotely located user based on a 
customized alert profile for the remotely located user, and wherein the customized alert profile 
includes previous performance information of the remotely located user, information to associate 
a level of difficulty of a particular test question with a capability category of the remotely located 
user to answer the particular test question, and a plurality of predetermined thresholds for the 
plurality of test questions," as recited in claims 6, 29, and 46. Therefore, the combination of 
Thomas, Sugimoto, and Hoehn-Saric does not teach or suggest these features. 

In view of the above, Appellants respectfully request withdrawal of the rejection of 
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independent claims 6, 29, and 46 under 35 U.S.C. § 103(a). Additionally, Thomas, Sugimoto, and 
Hoehn-Saric, taken individually or in combination, do not teach or suggest the features of 
dependent claims 12, 17-18, 32-33, 35, and 40-41 at least by virtue of their dependency on 
independent claims 6 and 29, respectively. Accordingly, Appellants respectfully request 
withdrawal of the rejection of claims 6, 12, 17-18, 29, 32-33, 35, 40-41, and 46 under 35 U.S.C. 
§103(a). 

C. 2. Claims 17 and 40 

In addition to the above, Appellants respectfully submit that claims 17 and 40 are 
independently distinguishable from the Thomas, Sugimoto, and Hoehn-Saric references. Claim 
17 depends from claim 6 and claim 40 depends from claim 29. Claims 17 and 40 additionally 
recite "monitoring the test question timing data for evidence of greater than expected response 
time to the test question, wherein outputting the test question timing data to a proctor device is 
performed by a test administration system in response to determining that evidence of greater 
than expected response time to the test question is present." Thomas, Sugimoto, and Hoehn- 
Saric, taken alone or in combination, do not teach or suggest this feature. To the contrary, a 
constant display of an elapsed time at an examinee device is disclosed. Thomas ' invention does 
not provide any control as to how a computer system executes a constant display. Appellents 
respectfully disagree that a constant display of elapsed time is the same as outputting test 
question timing data in response to determining that evidence of greater than expected response 
time to the test question is present. 

D. GROUND OF REJECTION 4 (Claims 7-8, 13, 16, 30-31, 36, and 39) 

The Final Office Action rejects claims 7-8, 13, 16, 30-31, 36, and 39 under 35 U.S.C. § 
103(a) as being unpatentable over Thomas in view of Sugimoto and Hoehn-Saric as applied to 
the claims above and further in view of APA. 

Additionally, Thomas and Hoehn-Saric, taken individually or in combination, do not 
teach or suggest the features of dependent claims 7-8, 13, 16, 30-31, 36, and 39 at least by virtue 
of their dependency on independent claims 6 and 29, respectively. Accordingly, Appellants 
respectfully request withdrawal of the rejection of claims 7-8, 13, 16, 30-31, 36, and 39 under 35 
U.S.C. § 103(a). 
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E. GROUND OF REJECTION 5 (Claims 14-15, 21-22, 37-38, and 44-45) 

The Final Office Action rejects claims 14-15, 21-22, 37-38, and 44-45 under 35 U.S.C. § 
103(a) as being unpatentable by Thomas in view of Sugimoto and Hoehn-Saric as applied to 
claim 6 above, and further in view of Kershaw et ah, System and Methods for Computer Based 
Testing , U.S. Patent No. 5,827,070 (October 27, 1998), hereinafter referred to as Kershaw. This 
rejection is respectfully traversed. 

E.l. Claims 14-15, 21-22, 37-38, and 44-45 

Since claims 14, 15, 21, 22, 37, 38, 44 and 45 depend from independent claims 6 and 29, 
the same distinctions between Thomas, Sugimoto, and Hoehn-Saric and the invention recited in 
claims 6 and 29 apply to dependent claims 14, 15, 21, 22, 37, 38, 44 and 45. As discussed above, 
Thomas, Sugimoto, and Hoehn-Saric, taken alone or in combination, fail to teach or suggest 
"alerting the remotely located user when the test question timing data exceeds a predetermined 
threshold based on an alert schedule for the test question while the remotely located user 
continues to attempt to answer the test question, wherein the alert schedule is generated for the 
remotely located user based on a customized alert profile for the remotely located user, and 
wherein the customized alert profile includes previous performance information of the remotely 
located user, information to associate a level of difficulty of a particular test question with a 
capability category of the remotely located user to answer the particular test question, and a 
plurality of predetermined thresholds for the plurality of test questions," as recited in claims 6, 
29, and 46. Kershaw is direct to a system and method for computer based testing. Kershaw 
discloses that, after all of the examinees' tests are graded, statistical processing may be provided 
to evaluate tests and test results. Kershaw shows an examinee database file that contains 
information such as test identification information, examinee registration information, and a 
session start time and a session end time for a test. Kershaw does not mention alerts for test 
questions or alert thresholds for test questions. The Office Action states that Kershaw lacks in 
tracking of question timing data. Kershaw does not provide for the deficiencies of Thomas, 
Sugimoto, and Hoehn-Saric with regard to independent claims 6, 29, and 46. Accordingly, 
Appellants respectfully request withdrawal of the rejection of claims 14, 15, 21, 22, 37, 38, 44 
and 45 under 35 U.S.C. §103(a) at least by virtue of their dependency on claims 6 and 29, 
respectively. 
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E.2. Claims 21-22 and 44-45 

In addition to being dependent on their respective independent claims, amended claims 
21-22 and 44-45 are also distinguished over the Thomas, Sugimoto, Hoehn-Saric, and Kershaw 
references based on the specific features recited therein. Claims 21-22 are dependent on 
independent claim 6; and claims 44-45 are dependent on independent claim 29. Thomas, 
Sugimoto, Hoehn-Saric, and Kershaw, taken individually or in combination, do not teach or 
suggest "storing a response to the test question from the remotely located user to update the 
customized alert profile for use in future tests," as recited in claims 21 and 44; and, in addition, 
"storing of the timing data for the test question to update timing data for the remotely located 
user in the customized alert profile for use in future tests," as recited in claims 22 and 45. 



F. GROUND OF REJECTION 6 (Claim 51) 

The Final Office Action rejects claim 51 under 35 U.S.C. § 103(a) as being unpatentable 
by Kershaw in view of Thomas, Hoehn-Saric, and Sugimoto. This rejection is respectfully 
traversed. 

Claim 51 reads as follows: 

51. A computer-implemented method for monitoring responses to test 
questions presented in a data processing system, the method comprising the steps 
of: 

administering, from an examination server, a plurality of tests to a plurality 
of remotely located users on a plurality of user devices and for each test of said 
plurality of tests that is administered: 

establishing a session identification for the administration of the 

test to the remotely located user, wherein said session identification 

includes a user identification, a test identifier, and a proctor device 

identifier; 

identifying presentation of the test questions on a user device of 
said user devices; 

responsive to the presentation of the test questions on said user 
device, monitoring test question timing data in which the test question 
timing data represents an elapsed time since an answered question from 
the test questions has been presented, wherein the elapsed time is an 
amount of time in attempting to answer a test question; 

correlating the test question timing data to the administration of the 
test to the remotely located user based on the session identification; 

wherein the test question timing data is output to said proctor 
device, based on said proctor device identifier, in response to determining 
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that evidence of greater than expected response time to the test question is 
present; and 

generating an alert message after the test question timing data 
exceeds a threshold while continuing to present the test question for the 
remotely located user to answer, wherein the alert message apprises the 
remotely located user that the elapsed time is excessive for the test 
question, wherein the alert message is generated based on an alert schedule 
for the test question, wherein the alert schedule is generated for the 
remotely located user based on a customized alert profile for the remotely 
located user, and wherein the customized alert profile includes previous 
performance information of the remotely located user, information to 
associate a level of difficulty of a particular test question with a capability 
category of the remotely located user to answer the particular test question, 
and a plurality of alert thresholds for the test questions ; 

wherein the remotely located user can send instant messages to and 
receive instant messages from a proctor device associated with said 
examination server and wherein said proctor device can send instant 
messages to and receive instant messages from the plurality of remotely 
located users, and wherein instant messages are used to communicate and 
clarify test question wording details, test instructions, and the test question 
timing data for the remotely located user during the test, (emphasis added) 

The Examiner bears the burden of establishing a prima facie case of obviousness based 
on the prior art when rejecting claims under 35 U.S.C. § 103. In re Fritch, 972 F.2d 1260, 23 
U.S.P.Q.2d 1780 (Fed. Cir. 1992). For an invention to be prima facie obvious, the prior art must 
teach or suggest all claim limitations. In re Royka, 490 F.2d 981, 180 USPQ 580 (CCPA 1974). 

Similarly, as discussed above, Kershaw, Thomas, Hoehn-Saric, and Sugimoto, taken 
alone or in combination, fail to teach or suggest "generating an alert message after the test 
question timing data exceeds a threshold while continuing to present the test question for the 
remotely located user to answer, wherein the alert message apprises the remotely located user 
that the elapsed time is excessive for the test question, wherein the alert message is generated 
based on an alert schedule for the test question, wherein the alert schedule is generated for the 
remotely located user based on a customized alert profile for the remotely located user, and 
wherein the customized alert profile includes previous performance information of the remotely 
located user, information to associate a level of difficulty of a particular test question with a 
capability category of the remotely located user to answer the particular test question, and a 
plurality of alert thresholds for the test questions." Accordingly, Appellants respectfully request 
withdrawal of the rejection of claim 51 under 35 U.S.C. § 103(a). 
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G. CONCLUSION 

As shown above, Kershaw, Thomas, Hoehn-Saric, and Sugimoto do not anticipate the 
claims. Similarly, the Examiner has failed to state a prima facie obviousness rejection against the 
claims. Therefore, Applicants request that the Board of Patent Appeals and Interferences reverse 
the rejections. Additionally, Applicants request that the Board direct the Examiner to allow the 
claims. 



/Gerald H. Glanzman/ 
Gerald H. Glanzman 
Reg. No. 25,035 
Yee & Associates, P.C. 
PO Box 802333 
Dallas, TX 75380 
(972) 385-8777 
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CLAIMS APPENDIX 



The text of the claims involved in the appeal are: 

1 . A computer-implemented method for monitoring responses to test questions presented in 
a data processing system, the method comprising the computer implemented steps of: 

identifying presentation of the test questions on the data processing system; 

responsive to the presentation of the test questions on the data processing system, 
monitoring test question timing data in which the test question timing data represents an elapsed 
time since an answered question from the test questions has been presented, wherein the elapsed 
time is an amount of time in attempting to answer a test question; and 

generating an alert after the test question timing data exceeds a threshold while 
continuing to present the test question for the test taker to answer, wherein the alert apprises a 
test taker that the elapsed time is excessive for the test question, wherein the alert is generated 
based on an alert schedule for the test question, wherein the alert schedule is generated for the 
test taker based on a customized alert profile for the test taker, and wherein the customized alert 
profile includes previous performance information of the test taker, information to associate a 
level of difficulty of a particular test question with a capability category of the test taker to 
answer the particular test question, and alert thresholds for the test questions. 

2. The computer-implemented method of claim 1, wherein the monitoring step is performed 
by a proctor device or a program on the data processing system. 
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3. The computer-implemented method of claim 1, wherein the presentation of the test 
questions is based on levels of difficulty of the test questions and the capability category of the 
test taker to answer the test questions. 

4. The computer-implemented method of claim 1 further comprising: 
billing a client for monitoring the presentation of the test questions. 

5 . The computer-implemented method of claim 1 , wherein the test questions are part of a 
test and further comprising: 

storing an identification of a number of test takers for the test; and 
billing a client based on the number of test takers for the test. 

6. A computer-implemented method of monitoring a test question response time, 
comprising the steps of: 

administering a test to a remotely located user of a client device; 

receiving test question timing data from the client device, the test question timing data 
representing an elapsed time used by the remotely located user in attempting to answer a test 
question from a plurality of test questions that are to be provided to the client device during 
administration of the test; 

outputting the test question timing data to a proctor device such that the proctor device 
may monitor the elapsed time in attempting to answer the test question for the remotely located 
user; 

wherein said remotely located user can send an instant message to and receive an instant 
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message from said proctor device and wherein said proctor device can send an instant message to 
and receive an instant message from a plurality of remotely located users, and wherein instant 
messages are used to communicate and clarify test question wording details, test instructions, and 
the test question timing data during the test; and 

alerting the remotely located user when the test question timing data exceeds a 
predetermined threshold based on an alert schedule for the test question while the remotely 
located user continues to attempt to answer the test question, wherein the alert schedule is 
generated for the remotely located user based on a customized alert profile for the remotely 
located user, and wherein the customized alert profile includes previous performance information 
of the remotely located user, information to associate a level of difficulty of a particular test 
question with a capability category of the remotely located user to answer the particular test 
question, and a plurality of predetermined thresholds for the plurality of test questions. 

7. The computer-implemented method of claim 6, further comprising billing a test developer 
for administration of the test to the remotely located user. 

8. The computer-implemented method of claim 6, further comprising billing the remotely 
located user for administration of the test. 

12. The computer-implemented method of claim 6, further comprising storing a score for the 
test in a permanent storage. 
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13. The computer-implemented method of claim 6, wherein the test is developed by a test 
developer and wherein the method is implemented by a test administration system that is 
operated by a different entity from the test developer. 

14. The computer-implemented method of claim 6, further comprising: 
receiving a request for administration of the test to the remotely located user; 
establishing a session identification for the administration of the test to the remotely 

located user; and 

correlating the test question timing data to the administration of the test to the remotely 
located user based on the session identification. 

15. The computer-implemented method of claim 14, wherein the session identification 
includes a proctor device identifier, and wherein outputting the test question timing data to the 
proctor device is based on the proctor device identifier. 

1 6. The computer-implemented method of claim 6, further comprising: 
storing an indicator of a number of test takers for the test; and 

billing a test developer of the test based on the number of test takers for the test. 

17. The computer-implemented method of claim 6, further comprising: 

monitoring the test question timing data for evidence of greater than expected response 
time to the test question, wherein outputting the test question timing data to a proctor device is 
performed by a test administration system in response to determining that evidence of greater 
than expected response time to the test question is present. 
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18. The computer-implemented method of claim 1 7, wherein monitoring the test question 
timing data for evidence of greater than expected response time to the test question includes 
comparing previously received test question timing data to currently received test question timing 
data to determine if a change in the test question timing data indicates evidence of greater than 
expected response time to the test question. 

2 1 . The computer-implemented method of claim 6, further comprising: storing a response to 
the test question from the remotely located user to update the customized alert profile for use in 
future tests. 

22. The computer-implemented method of claim 6, further comprising: storing of the timing 
data for the test question to update timing data for the remotely located user in the customized 
alert profile for use in future tests. 

23. An apparatus for monitoring responses to test questions presented in a data processing 
system, the apparatus comprising: 

identifying means for identifying presentation of the test questions on the data processing 

system; 

monitoring means, responsive to the presentation of the test questions on the data 
processing system, for monitoring test question timing data in which the test question timing data 
represents an elapsed time since an answered question from the test questions has been presented, 
wherein the elapsed time is an amount of time in attempting to answer a test question; and 

generating means for generating an alert after the test question timing data exceeds a 
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threshold while continuing to present the test question for the test taker to answer, wherein the 
alert apprises a test taker that the elapsed time is excessive for the test question, wherein the alert 
is generated based on an alert schedule for the test question, wherein the alert schedule is 
generated for the test taker based on a customized alert profile for the test taker, and wherein the 
customized alert profile includes previous performance information of the test taker, information 
to associate a level of difficulty of a particular test question with a capability category of the test 
taker to answer the particular test question, and alert thresholds for the test questions. 

24. The apparatus of claim 23, wherein the apparatus is a proctor device or a program on the 
data processing system. 

25. The apparatus of claim 23, wherein the presentation of the test questions is based on 
levels of difficulty of the test questions and the capability category of the test taker to answer the 
test questions. 

26. The apparatus of claim 23 further comprising: 

billing means for billing a client for monitoring the presentation of the test questions. 

27. The apparatus of claim 23, wherein the test questions are part of a test and further 
comprising: 

storing means for storing an identification of a number of test takers for the test; and 
billing means for billing a client based on the number of test takers for the test. 
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28. A data processing system comprising: 
a bus system; 

a communications unit connected to the bus system; 

a storage device connected to the bus system, wherein the storage device includes a set of 
instructions; and 

a processing unit connected to the bus system, wherein the processing unit executes the 
set of instructions to identify presentation of the test questions on the data processing system, 
monitor test question timing data in response to the presentation of the test questions on the data 
processing system in which the test question timing data represents an elapsed time since an 
answered question from the test questions has been presented, wherein the elapsed time is an 
amount of time in attempting to answer a test question, and generate an alert after the test 
question timing data exceeds a threshold while continuing to present the test question for a test 
taker to answer, wherein the alert apprises the test taker that the elapsed time is excessive for the 
test question, wherein the alert is generated based on an alert schedule for the test question, 
wherein the alert schedule is generated for the test taker based on a customized alert profile for 
the test taker, and wherein the customized alert profile includes previous performance 
information of the test taker, information to associate a level of difficulty of a particular test 
question with a capability category of the test taker to answer the particular test question, and 
alert thresholds for the test questions. 

29. An apparatus for monitoring a test question response time, comprising: 
a controller; and 
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at least one interface coupled to the controller, wherein the controller administers a test to 
a remotely located user of a client device via the at least one interface, receives test question 
timing data from the client device via the at least one interface, the test question timing data 
representing an elapsed time used by the remotely located user in attempting to answer a test 
question from a plurality of test questions that are to be provided to the client device during 
administration of the test, outputs the test question timing data to a proctor device via the at least 
one interface, such that the proctor device may monitor the elapsed time in attempting to answer 
the test question for the remotely located user, wherein said processing unit provides instant 
messaging between said remotely located user and a proctor, wherein said remotely located user 
can send an instant message to and receive an instant message from said proctor and wherein said 
proctor can send an instant message to and receive an instant message from a plurality of 
remotely located users, and wherein instant messages are used to communicate and clarify test 
question wording details, test instructions, and the test question timing data during the test, and 
wherein the controller alerts the remotely located user when the test question timing data exceeds 
a predetermined threshold based on an alert schedule for the test question, wherein the alert 
schedule is generated for the remotely located user based on a customized alert profile for the 
remotely located user, and wherein the customized alert profile includes previous performance 
information of the remotely located user, information to associate a level of difficulty of a 
particular test question with a capability category of the remotely located user to answer the 
particular test question, and a plurality of predetermined thresholds for the plurality of test 
questions. 
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30. The apparatus of claim 29, wherein the controller bills a test developer for administration 
of the test to the remotely located user. 

3 1 . The apparatus of claim 29, wherein the controller bills the remotely located user for 
administration of the test. 

32. The apparatus of claim 29, wherein the controller sends an instant message to the client 
device via the at least one interface. 

33 . The apparatus of claim 29, wherein the controller receives an instant message from the 
client device via the at least one interface. 

35. The apparatus of claim 29, further comprising a storage device, wherein the controller 
stores a score for the test in the storage device. 

36. The apparatus of claim 29, wherein the test is developed by a test developer and wherein 
the apparatus is operated by a different entity from the test developer. 

37. The apparatus of claim 29, wherein the controller receives a request for administration of 
the test to the remotely located user, establishes a session identification for the administration of 
the test to the remotely located user, and correlates the test question timing data to the 
administration of the test to the remotely located user based on the session identification. 
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38. The apparatus of claim 37, wherein the session identification includes a proctor device 
identifier, and wherein the controller outputs the test question timing data to the proctor device 
based on the proctor device identifier. 

39. The apparatus of claim 29, further comprising a storage device, wherein the controller 
stores an indicator of a number of test takers for the test in the storage device and bills a test 
developer of the test based on the number of test takers for the test. 

40. The apparatus of claim 29, wherein the controller monitors the test environment data for 
evidence of greater than expected response time to the test question, and wherein the controller 
outputs the test question timing data to a proctor device in response to determining that evidence 
of greater than expected response time to the test question is present. 

41 . The apparatus of claim 40, wherein the controller monitors the test question timing data 
for evidence of greater than expected response time to the test question by comparing previously 
received test question timing data to currently received test environment data to determine if a 
change in the test question timing data indicates evidence of greater than expected response time 
to the test question. 

44. The apparatus of claim 29, further comprising: storing a response to the test question 
from the remotely located user to update the customized alert profile for use in future tests. 

45. The apparatus of claim 29, further comprising: storing of the timing data for the test 
question to update timing data for the remotely located user in the customized alert profile for 
use in future tests. 
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46. A computer program product in a computer readable medium for monitoring a test 
question response time, comprising: 

first instructions for administering a test to a remotely located user of a client device; 

second instructions for receiving test question timing data from the client device, the test 
question timing data representing an elapsed time used by the remotely located user in attempting 
to answer a test question from a plurality of test questions that are to be provided to the client 
device during administration of the test; 

third instructions for outputting the test question timing data to a proctor device such that 
the proctor device may monitor the elapsed time in attempting to answer the test question for the 
remotely located user; and 

fourth instructions for providing instant messaging between the remotely located user and 
a proctor, wherein said remotely located user can send an instant message to and receive an 
instant message from said proctor and wherein said proctor can send an instant message to and 
receive an instant message from a plurality of remotely located users, and wherein instant 
messages are used to communicate and clarify test question wording details, test instructions, and 
the test question timing data during the test; and 

fifth instructions for alerting the remotely located user when the test question timing data 
exceeds a predetermined threshold based on an alert schedule for the test question while the 
remotely located user continues to attempt to answer the test question, wherein the alert schedule 
is generated for the remotely located user based on a customized alert profile for the remotely 
located user, and wherein the customized alert profile includes previous performance information 
of the remotely located user, information to associate a level of difficulty of a particular test 
question with a capability category of the remotely located user to answer the particular test 
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question, and a plurality of predetermined thresholds for the plurality of test questions. 

47. A computer program product in a computer readable medium for use in monitoring 
responses to test questions presented in a data processing system, the computer program product 
comprising: 

first instructions for identifying presentation of the test questions on the data processing 

system; 

second instructions, responsive to the presentation of the test questions on the data 
processing system, for monitoring test question timing data in which the test question timing data 
represents an elapsed time since an answered question from the test questions has been 
presented, wherein the elapsed time is an amount of time in attempting to answer a test question; 
and 

third instructions for generating an alert after the test question timing data exceeds a 
threshold while continuing to present the test question for the test taker to answer, wherein the 
alert apprises a test taker that the elapsed time is excessive for the test question, wherein the alert 
is generated based on an alert schedule for the test question, wherein the alert schedule is 
generated for the test taker based on a customized alert profile for the test taker, and wherein the 
customized alert profile includes previous performance information of the test taker, information 
to associate a level of difficulty of a particular test question with a capability category of the test 
taker to answer the particular test question, and alert thresholds for the test questions. 

48. The computer program product of claim 47, wherein the second instructions are located 
in a proctor device or a program on the data processing system. 
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49. The computer program product of claim 47 further comprising: 

fourth instructions for billing a client for monitoring the presentation of the test questions. 

50. The computer program product of claim 47, wherein the test questions are part of a test 
and further comprising: 

fourth instructions for storing an identification of a number of test takers for the test; and 

fifth instructions for billing a client based on the number of test takers for the test. 

51. A computer-implemented method for monitoring responses to test questions presented in 
a data processing system, the method comprising the steps of: 

administering, from an examination server, a plurality of tests to a plurality of 
remotely located users on a plurality of user devices and for each test of said plurality of 
tests that is administered: 

establishing a session identification for the administration of the test to the 
remotely located user, wherein said session identification includes a user 
identification, a test identifier, and a proctor device identifier; 

identifying presentation of the test questions on a user device of said user 
devices; 

responsive to the presentation of the test questions on said user device, 
monitoring test question timing data in which the test question timing data 
represents an elapsed time since an answered question from the test questions has 
been presented, wherein the elapsed time is an amount of time in attempting to 
answer a test question; 
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correlating the test question timing data to the administration of the test to 
the remotely located user based on the session identification; 
wherein the test question timing data is output to said proctor device, based on said proctor 
device identifier, in response to determining that evidence of greater than expected response 
time to the test question is present; and 

generating an alert message after the test question timing data exceeds a 
threshold while continuing to present the test question for the remotely located 
user to answer, wherein the alert message apprises the remotely located user that 
the elapsed time is excessive for the test question, wherein the alert message is 
generated based on an alert schedule for the test question, wherein the alert 
schedule is generated for the remotely located user based on a customized alert 
profile for the remotely located user, and wherein the customized alert profile 
includes previous performance information of the remotely located user, 
information to associate a level of difficulty of a particular test question with a 
capability category of the remotely located user to answer the particular test 
question, and a plurality of alert thresholds for the test questions; 
wherein the remotely located user can send instant messages to and receive instant messages 
from a proctor device associated with said examination server and wherein said proctor 
device can send instant messages to and receive instant messages from the plurality of 
remotely located users, and wherein instant messages are used to communicate and clarify 
test question wording details, test instructions, and the test question timing data for the 
remotely located user during the test. 
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EVIDENCE APPENDIX 



There is no evidence to be presented. 
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RELATED PROCEEDINGS APPENDIX 



There are no related proceedings. 
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